behavior as their predictability grows 
during the course of learning. The dif- 
ference between the actual occurrence 
and the prediction of the reinforcer is 
denoted as the reinforcement predic- 
tion error signal. In what is known in 
the art as the temporal-difference 
model (TD model) of Pavlovian learn- 
ing, the reinforcement prediction 
error signal is used to learn a rein- 
forcement prediction signal. The de- 
sired reinforcement prediction signal 
is defined as a weighted sum of future 
reinforcement signals wherein rein- 
forcements are progressively dis- 
counted with increasing time into the 
future. The reinforcement prediction 
error signal progressively decreases 
during learning as the reinforcement 
prediction signal becomes more simi- 
lar to the desired reinforcement pre- 
diction signal. 

•Algorithms based on the TD model 
("TD algorithms") have been analyzed 
and shown to be implementations of a 
variant of dynamic programming. Ma- 
chine-learning studies have shown that 
TD algorithms are powerful means of 
solving reinforcement learning prob- 
lems that involve delayed reinforce- 


ment. Examples of such problems in- 
clude board games, which involve de- 
layed rewards (winning) or punish- 
ments (losing). 

•Mid-brain dopamine neurons are so 
named because they modulate the lev- 
els of activity of other neurons by 
means of the n euro transmitter chemi- 
cal dopamine. The cell bodies of 
dopamine neurons are located in the 
brain stem and their axons project to 
many brain areas. Activities of 
dopamine neurons have been found 
to be strikingly similar to the predic- 
tion error signals of the TD model. 

• Real neural signals include spikelike 
pulses, the times of occurrence of which 
are significant. The term "biological 
spike coding" denotes, essentially, tem- 
poral labeling of such pulses in real neu- 
rons or in a neural-network model. 

This concludes the background infor- 
mation. 

The present neural-network model 
incorporates biological spike coding 
along with some basic principles of the 
learning by synapses in the cortex of 
the human brain. According to the 
learning rule of the model, synaptic 
weights are adapted when pre- and 


postsynaptic spikes occur within short 
time windows. In simplified terms, for a 
given synapse and time window, the 
synaptic strength is increased in the 
long term if the presynaptic spike pre- 
cedes the postsynaptic spike or is de- 
creased in the long term if the presy- 
naptic spike follows the postsynaptic 
spike. This learning rule has been 
shown to minimize prediction errors, 
indicating that the neural network 
learns an optimal dynamic model of an 
external process. 

This work was done by Tuan Duong, Vu 
Duong, and Roland Suri of Caltech for 
NASA's Jet Propulsion La bora tory. 
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©Evolutionary Computing Methods for Spectral Retrieval 

Solutions to the inverse problem of spectral retrieval are found in a computationally efficient 
process. 

N ASA's J& Propulsion Laboratory, Pasadena, California 


A methodology for processing spec- 
tral images to retrieve information on 
underlying physical, chemical, and/or 
biological phenomena is based on evolu- 
tionary and related computational 
methods implemented in software. In a 
typical case, the solution (the informa- 
tion that one seeks to retrieve) consists 
of parameters of a mathematical model 
that represents one or more of the phe- 
nomena of interest. 

The methodology was developed for 
the initial purpose of retrieving the 
desired information from spectral 
image data acquired by remote-sens- 
ing instruments aimed at planets (in- 
cluding the Earth). Examples of infor- 
mation desired in such applications 
include trace gas concentrations, tem- 
perature profiles, surface types, 
day/ night fractions, cloud/ aerosol 
fractions, seasons, and viewing angles. 
The methodology is also potentially 
useful for retrieving information on 


chemical and/ or biological hazards in 
terrestrial settings. 

In this methodology, one utilizes an it- 
erative process that minimizes a fitness 
function indicative of the degree of dis- 
similarity between observed and syn- 
thetic spectral and angular data. The 
evolutionary computing methodsthat lie 
at the heart of this process yield a popu- 
lation of solutions (sets of the desired pa- 
rameters) within an accuracy repre- 
sented by a fitness-function value 
specified by the user. The evolutionary 
computing methods (ECM) used in this 
methodology are Genetic Algorithms 
and Simulated Annealing, both of which 
are well-established optimization tech- 
niques and have also been described in 
previous NASA Tech Briefs articles. These 
are embedded in a conceptual frame- 
work, represented in the architecture of 
the implementing software, that enables 
automatic retrieval of spectral and angu- 
lar data and analysis of the retrieved solu- 


tions for uniqueness. This framework is 
composed of three modules (see figure): 

1. The central core, which consistsof the 
aforementioned ECM; 

2. The synthetic-spectra generator, 
which, coupled with the ECM, gener- 
ates a population of automatically re- 
trieved spectral solutions; and 

3. The synthetic-spectra degeneracy ana- 
lyzer ("degeneracy" is used here in 
the mathematical sense of signifying 
the existence of multiple equally valid 
solutions for a given set of data and 
user-defined accuracy), which applies 
several well-established mathematical 
methods to characterize the unique- 
ness (or the degeneracy, which is es- 
sentially the lack of uniqueness) of 
the solutions within the population. 
One advantage afforded by this ECM- 

based methodology over traditional spec- 
tral retrieval methods is the ability to per- 
form an automatic, unbiased search for 
all solutions within the entire parameter 
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This Conceptual Framework for Spectral Retrieval, and the software that implements it, comprises 
three modules that interact in an iterative process. 


space, using criteria that make searching 
computationally far more economical 
than in complete-enumeration ("brute 
force"), Monte Carlo, or random 


searches. (As used here, "unbiased" char- 
acterizes a search that does not depend 
on initial ad hoc guesses by experts.) 
Other advantages include the following: 


• Optimal solutions are found; 

• Better interpretations of planetary spec- 
tral and angular data are possible, and 
initial tests have shown these interpreta- 
tions to be consistent with ground 
truth; and 

• The methodology is not limited to spe- 
cific problems, and can be extended to 
solve problems of greater complexity. 

T his work was done by R i chard Terri 1$ Wolf- 
gang Fink, TerranceH untsberger, Seungwon Lee 
Edwin Tisdale Paul Von Allmen, andGiovanna 
Tinetti of Caltech for NASA's jet Propul si on Labo- 
ratory. Further information is contained in a 
TSP (see page 1). 
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© Monitoring Disasters by Use of Instrumented Robotic Aircraft 

Real-time synoptic data would help in coordinating and planning responses. 

AmesResearch Center, M off ett Field, California 


Efforts are under way to develop data- 
acquisition, data-processing, and data- 
communication systems for monitoring 
disasters over large geographic areas by 
use of uninhabited aerial systems (U AS) 
— robotic aircraft that are typically pi- 
loted by remote control. As integral 
parts of advanced, comprehensive disas- 
ter-management programs, these sys- 
tems would provide (1) real-time data 
that would be used to coordinate re- 
sponses to current disasters and (2) 
recorded data that would be used to 
model disasters for the purpose of miti- 
gating the effects of future disasters and 
planning responses to them. 

The basic idea is to equip UAS with 
sensors (e.g., conventional video cameras 
and/or multispectral imaging instru- 
ments) and to fly them over disaster 
areas, where they could transmit data by 
radio to command centers. Transmission 
could occur along direct line-of-sight 
paths and/or along over-the-horizon 
paths by relay via spacecraft in orbit 
around the Earth. The initial focus is on 


demonstrating systems for monitoring 
wildfires; other disasters to which these 
developments are expected to be applica- 
ble include floods, hurricanes, tornadoes, 
earthquakes, volcanic eruptions, leaks of 
toxic chemicals, and military attacks. 

The figure depicts a typical system for 
monitoring a wildfire. In thiscase, instru- 
ments aboard a UAS would generate cal- 
ibrated thermal-infrared digital image 
data of terrain affected by a wildfire. The 
data would be sent by radio via satellite to 
a data-archive server and image-process- 
ing computers. In the image-processing 
computers, the data would be rapidly 
geo-rectified for processing by one or 
more of a large variety of geographic-in- 
formation-system (GIS) and/or image- 
analysis software packages. After process- 
ing by this software, the data would be 
both stored in the archive and distrib- 
uted through standard Internet connec- 
tions to a disaster-mitigation center, an 
investigator, and/ or command center at 
the scene of the fire. 

Ground assets (in thiscase, firefighters 


and/or firefighting equipment) would 
also be monitored in real time by use of 
Global Positioning System (GPS) units 
and radio communication links between 
the assets and the UAS. In this scenario, 
the UAS would serve as a data-relay sta- 
tion in the sky, sending packets of infor- 
mation concerning the locations of assets 
to the image-processing computer, 
wherein this information would be incor- 
porated into the geo-rectified images and 
maps. Hence, the images and maps 
would enable command-center person- 
nel to monitor locations of assets in real 
time and in relation to locations affected 
by the disaster. Optionally, in case of a dis- 
aster that disrupted communications, the 
UAS could be used as an airborne com- 
munication relay station to partly restore 
communications to the affected area. 

A prototype of a system of this type was 
demonstrated in a project denoted the 
First Response Experiment (Project 
FiRE). In this project, a controlled out- 
door fire was observed by use of a ther- 
mal multispectral scanning imager on a 
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